Kolmogorov width decay and poor approximators in machine learning: shallow neural networks, random feature models and neural tangent kernels

We establish a scale separation of Kolmogorov width type between subspaces given Banach space under the condition that sequence linear maps converges much faster on one subspaces. The general technique is then applied to show reproducing kernel Hilbert spaces are poor $$L^{2}$$ -approximators for class two-layer neural networks in high dimension, and multi-layer with small path norm approximators certain Lipschitz functions, also -topology.